Accepted for publication in Astrophysical Journal Letters 

Preprint typeset using I^T^X style cmulatcapj v. 10/09/06 



A CONFIRMATION OF THE STRONG CLUSTERING OF DISTANT RED GALAXIES AT 2 < Z < 3 
Ryan F. Quadri 1 , Rik J. Williams 1 , Kyoung-Soo Lee 2 , Marijn Franx 1 , Pieter van Dokkum 3 Gabriel 



B. Brammer 



Accepted for publication in Astrophysical Journal Letters 



oo 

o 

O 

(N 

< 



6 



q 

oo 
o 
oo 
o 



ABSTRACT 

Recent studies have shown that distant red galaxies (DRGs), which dominate the high-mass end of 
the galaxy population at z ~ 2.5, are more strongly clustered than the population of blue star-forming 
galaxies at similar redshifts. However these studies have been severely hampered by the small sizes of 
fields having deep near-infrared imaging. Here we use the large UKIDSS Ultra Deep Survey to study 
the clustering of DRGs. The size and depth of this survey allows for an unprecedented measurement 
of the angular clustering of DRGs at 2 < Zphot < 3 and K < 21. The correlation function shows the 
expected power law behavior, but with an apparent upturn at 9 < 10". We deproject the angular 
clustering to infer the spatial correlation length, finding 10.6±1.6/i -1 Mpc. We use the halo occupation 
distribution framework to demonstrate that the observed strong clustering of DRGs is not consistent 
with standard models of galaxy clustering, confirming previous suggestions that were based on smaller 
samples. Inaccurate photometric redshifts could artificially enhance the observed clustering, however 
significant systematic redshift errors would be required to bring the measurements into agreement 
with the models. Another possibility is that the underlying assumption that galaxies interact with 
their large-scale environment only through halo mass is not valid, and that other factors drive the 
evolution of the oldest, most massive galaxies at z ~ 2. 

Subject headings: cosmology: large-scale structure of the universe — galaxies: evolution — galaxies: 
formation — galaxies: high-redshift — infrared: galaxies 



1. INTRODUCTION 

Thus far, the precise measurements of galaxy cluster- 
ing at z > 2 that are necessary for meaningful phys- 
ical interpretation have only been possible for the rel- 
atively blue, s tar-forming galaxies that domina t e op- 
tical surveys (lAdelberger et alj l2005t iLee et al.l 120061 : 
lOuchi et alJ I2005T) . However the most massive galax- 
ies at these redshifts tend to be faint in the optical, 
and are more appropriately sel ected in the near-infrared 
(e.g. Ivan Dokkum et all I2006D . In particular, galax- 
ies meeting the J — K > 2 . 3 crit erion for distant red 
galaxies (DRGs; iFranx et alj 120031 ) have been shown to 
domi nate the high-mass end o f the galaxy mass func- 
tion (Ivan Dokkum et all 120061 : iMarchesini et alj l2007at 
iRudnick et alj I2006D . Measurements of DRG cluster- 
ing have been severely hampered by small fields and by 
the near-complete reliance on photometric redshifts. Us- 
ing the largest DRG sample then available. I Quadri et all 
(2007a) confirmed t he very strong cl u stering found 
by p revious authors (jDaddi et al.1 120031 : iGrazian et al.1 
2006). They also showed that strong clustering implies 
that DRGs occupy dark matter halos with M > 1O 13 M . 
However, the number density of these dark matter halos 
is at least an order of magnitude smaller than the num- 
ber density of DRGs, suggesting that the DRG cluster- 
ing measurements are incompatible with models of dark 
matter clustering. 

Here we use a -0.65deg 2 field from the UKIRT In- 



1 Leiden Observatory, Leiden University, NL-2300 RA, Leiden, 



Electronic address: quadri@strw.leidenuniv.nl 

, NL-: 

Netherlands 

2 Yale Center for Astronomy and Astrophysics, Yale University, 
New Haven, CT 06520 



frared Deep Sky Survey (UKID SS), which is ~8 t imes 
larger than the area used by I Quadri et al.l (|2007af ) but 
with similar NIR depth, to determine whether the strong 
observed clustering of DRGs found in previous studies 
was an artifact due to limited field sizes or whether some 
other explanation must be found. 

We use (fi M ,^A,cr 8 » = (0.3,0.7,0.9,0.7). Small 
changes in these parameters do not affect our basic con- 
clusions. Magnitudes are given in the Vega system, ex- 
cept where noted. 

2. DATA 

The UKIDSS project covers different areas to different 
depths; here we make use of the deepest UKIDSS dataset, 
known as the Ultra Dee p Survey (UPS). We use the UDS 
Data Release 1 images ([Warren et al.ll2007t ). which reach 
5 a point-source dep ths of J ~ 23 and K ~ 21.6. We note 
that lFoucaud et alJ (|2007h used the UDS Early Data Re- 
lease to study the clustering of DRGs down to K ~ 19, 
however such bright DRGs lie primarily at z < 2 and 
may not be directly relevant to the galaxies that are the 
subject of this work. Most of the UDS field has coverage 
in the optical bands from the Subaru-XMM deep Survey 
(SXDS) 4 . We use the beta-release of the BRi'z' images, 
which reach depths of ~25.3-27.5 (AB magnitudes). Fi- 
nally, we combine these data with 3.6/z and 4.5/i imag- 
ing from the Spi tzer Wide- Area Infra red Extragalactic 
Survey (SWIRE: ILonsdale et al1l2003l) . The procedures 
used to create a multicolor if -selected catalog from these 
imaging data are detailed by I Williams et al.l(|2008f) . 
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3. PHOTOMETRIC REDSHIFTS 

4 http:/ /www. naoj.org/Science/SubaruProject/SXDS/ 
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Accurate redshift information is fundamentally im- 
portant for clustering studies. Photometric red- 
shifts were calculated using the EA ZY software 
(|Brammer. van Dokkum. fc Coppi I I2008D 5 . EAZY fits 
linear combinations of galaxy templates to the observed 
photometry using a Bayesian prior for the distribution 
of redshifts as a function of apparent magnitude. The 
template set was carefully chosen to provide high-quality 
photometric redshifts for fC-selected galaxies in several 
current deep surveys. 

To assess the accuracy of our photometric redshifts we 
compare to a sample 119 spectroscopic re dshifts available 
in th is field (for further description, see William s et al.l 
2008). The normalized median absolute deviation of 
Az/(1 + z) is onmad = 0.033. As nearly all spectro- 
scopic redshifts in this field are at z < 1.5, and none 
are for DRGs, we use public data on the well-studied 
Chandra Deep Field-South (CDF-S) to obtain an esti- 
mate of the expected photometric redshift accuracy for 
the galaxies currently under consideration . We use the 
photometric catalog of IWuvts et al.l (|2008f) . and exclude 
the UVH filters in order to match the filter set available 
in the UDS. Restricting the sample to the DRGs in CDF- 
S with spectroscopic redshifts, we find onmad = 0.065, 
with no apparent dependence on DRG redshift and neg- 
ligible systematic offsets. This suggests that the DRG 
photometric redshifts in our study also have an accuracy 
of ctnmad ~ 0.06 — 0.07, although this can only be taken 
as a rough indication. 

The overall redshift distribution of the sample is used 
to deproject the observed angular correlation function in 
order to infer the spatial correlation function. This dis- 
tribution is often estimated simply using a histogram of 
the photometric redshifts, without taking into account 
the redshift uncertainties. Because we select DRGs in 
a specific photometric redshift range, it is expected that 
random errors will cause galaxies to scatter into the sam- 
ple from both lower and higher redshifts. It is also ex- 
pected that galaxies within the redshift selection window 
will scatter out, however this will not affect the cluster- 
ing so long as such galaxies are drawn randomly from 
the sample (systematic errors which cause galaxies to 
scatter into or out of the selection window are much 
more problematic). To estimate the intrinsic redshift 
distribution of the galaxies in our sample, we use Monte 
Carlo simulations in which the observed photometry is 
perturbe d according the the photometric uncertainties 
(see also lQuadri et al.ll2007af l. This procedure broadens 
the distribution relative to what would be inferred from a 
histogram of photometric redshifts, resulting in a higher 
correlation length; we return to this point in § [S] 

4. MEASUREMENTS OF THE DRG CORRELATION 
FUNCTION 

For a detailed description of the techniques used 
to perform c orrelat ion function measurements, see 
iQuadri et al.1 (|2007a[ ). Here we briefly describe the 
method, and point out differences between the method 
used here and in the previous paper. The angular 
two-point correlation function is calculated using the 
lLandv fc Szalavl (|1993f ) estimator. Uncertainties are es- 
timated using bootstrap resampling, which yields un- 

5 http:/ /www. astro. yale.edu/eazy/ 



certainties that are significantly larger than those ex- 
pected from Pois s on sta tistics. We follow the method of 
lAdelberger et alj |2005) to correct for the integral con- 
straint, although the correction is small given the large 
size of the field. 

A potentially serious problem for measurements of the 
auto-correlation function arises from variable sensitivity 
or calibration errors across the images. We measured 
the correlation function of stars in order to test whether 
such variations induce artificial clustering, obtaining the 
expected result that stars are unclustered. We have also 
measured the clustering of DRGs selected from catalogs 
in which the J and K zeropoints are varied within their 
uncertainties independently for each of the pointings that 
make up the UDS mosaic. Although it is difficult to rule 
out significant errors in the correlation function arising 
from such variations, it appears that they are not a dom- 
inant source of error. 

The redshift distribution of galaxies selected using the 
DRG color criterion is rather wide, extending from z ~ 1 
to at le ast z ~ 3.5 (e.g. lGrazian et all 2006; Quad rTet al.l 
l2007blh We therefore impose the additional criterion 
2 < Zphot < 3. Over this redshift range, the DRG crite- 
rion effecti vely selects galaxies with red rest-frame opti- 
cal colors (|Quadri et al.|[2007bf ). We also limit the sam- 
ple to objects with K < 21, ensuring that the catalog 
is complete and that objects have a sufficient signal-to- 
noise ratio to calculate accurate photometric redshifts. 
We visually inspected each object in the sample, reject- 
ing those that appeared to be artifacts in the images 
or were otherwise deemed to have unreliable photome- 
try. Objects were also rejected if the templates used for 
photometric redshifts gave poor fits. The final sample 
consists of 1528 DRGs with K < 21. 

Figure [1] shows the angular correlation function w(9) 
of DRGs. Our measurements confi rm the strong angular 
clus tering previously rep orted by iGrazian et alj (2006) 
and I Quadri et al.1 (|2007a( ). but with reduced uncertain- 
ties and extending to much larger angular scales. Imme- 
diately apparent is the departure of w{9) from a power 
law on small scales. To illustrate the significance of this 
excess clustering signal, and to show how the results dif- 
fer from what would be measured using a smaller field, 
we fit a power law A w 9~^ over the range 2" < 9 < 40" 
and over 40" < 9 < 500". The former fit yields 
(A w ,/3) = (12 ± 8, 1.2 ± 0.3) while the latter fit yields 
(A w ,/3) = (1.1 ±0.8, 0.47 ±0.14). Note that there is sig- 
nificant covariance between the two fitting parameters, so 
the uncertainties should be take n as representative only . 
As noted by p r evious author s (Adelber ger et al.1 120051 : 
iLee et al.1 [20061: IQuadri et aill2007al) . the large-scale fit 
is preferred when comparing galaxy samples, as it better 
probes the clustering of dark matter halos that host the 
galaxies. 

The bottom panel of Figured] shows the residuals w(9) 
from the large-scale power law. A small-scale excess has 
also been observed in large samples of optical l y selected 
galax ies at high redshift fe.g. lOuchi et al.|[2005l : lLee et al.1 
20061). and for var ious galaxy samples at < z < 2 (e.g. 
Zehavi et al.ll2005f ). 

The spatial correlation length rg can be e stimated from 
w(9) u sing the Limber projection; see e.g. IQuadri et al.l 
(|2007al) for further details. The result is r = 10.6 ± 
1.6/i _1 Mpc, where we have used the large-scale fitting 
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Fig. 1. — Top: The angular correlation function of DRGs with 
2 < Zphot < 3 and K < 21. The open circles illustrate the observed 
values, while the filled circles have been corrected for the integral 
constraint. Two power-law fits are shown. The gray dashed line 
indicates the best fitting power law over 2" < 9 < 40"; it is appar- 
ent that this power law does not provide a good fit over the entire 
observed range. The black dashed line indicates the best-fitting 
power law over 40" < 9 < 500". Bottom: The departure of w(9) 
from the 40" < 9 < 500" power law. 

parameters and the uncertainty is estimated using the 
bootstrap simulations described above. This uncertainty 
accounts for the uncertainty in 7, but not in the redshift 
distrib ution N(z). We use the method of lHamana et al.l 
(2004) to estimate the linear bias directly from w(9), 
finding b = 5.0 ± 0.4. 

5. THE HALO MODEL 

iQuadri et ail (|2007aD found that DRGs significantly 
outnumber the dark matter halos that are clustered 
strongly enough to host them. Models of the halo oc- 
cupation distribution (HOD) naturally account for this 
type of disc repancy by al l owing halos to host multiple 
galaxies, but lQuadri et al.l (|2007af l suggest that the large 
numbers of galaxies that would be required to share ha- 
los can be ruled out by the observed small-scale cluster- 
ing. Given that the DRG clustering results presented in 
this work are largely consistent with those from previous 
works, it is expected that the observations are still in 
conflict with the models. In this section we use an HOD 
model to show this explicitly. 

In the HOD framework, the galaxy correlation func- 
tion is understood to be the sum of two components. 
On large scales the correlation function follows that of 
the host dark matter halos. On small scales there is an 
additional contribution from galaxy pairs within individ- 
ual halos^_We follow the modeling procedures described 
by iLee et al.l (|2006t l. to which we refer the reader for 
details. Briefly, we calculate the num ber density and 
bias o f halos using the prescri p tions of ISheth fc Tormenl 
l[i99flh : ISheth. Mo. fc Torrnenl IpOOlh . The halo occupa- 
tion number, which describes the number of galaxies per 
halo, is parameterized as N occ (Mh) = (M^/Mi)" for halo 
mass Mh > M m i n and N occ (Mh) = otherwise. Thus 
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Fig. 2. — The angular correlation function and the best-fitting 
HOD models. The solid lines indicate the models, and the dot- 
ted lines indicate the contributions from galaxy pairs within single 
halos and within separate halos. The lower model provides the 
best fit over the range 2" < 9 < 500", with fitting parameters 
(M mi n,Mi,a) = (1.3 x 1O 12 M ,1.3 x 10 13 M©,0.9). The upper 
model provides the best fit over 50" < 9 < 500", with fitting pa- 
rameters (M min ,Mi,a) = (1.8 x 10 13 M Q ,1.3 x 10 12 M Q ,0.9). It 
is apparent that neither model adequately describes the data. 

there are three free parameters, Mi, M m i n , and a. If 
the number density n g of galaxies is known, then one 
of these parameters can be fixed for assumed values of 
the other two parameters. We obtain a rough estimate 
of n g pa 6.5 x 10 _4 ft. 3 Mpc -3 using the effective volume 
probed by our sample, which is determined using the red- 
shift selection function described in § [3l This density is 
in reasonable agreement with an independent estimate of 
n g pa (5.0 ± 0.9) x 10~ 4 /t 3 Mpc~ 3 , which i s based on the 
luminosity functions of Marchcs ini et al.l (|2007af ). 

Figure [2] shows two models chosen according to % 2 fits. 
The lower solid line shows a model that is fit over 2" < 
8 < 500". While this model provides an adequate fit on 
smaller angular scales, it systematically under-predicts 
the clustering on larger scales. To better illustrate the 
nature of the disagreement, it is useful to inspect the 
upper solid line, which shows a model that is fit only over 
50" < 8 < 500". While this model provides an adequate 
fit at larger scales, the small-scale fit is unacceptable. As 
already noted, the fundamental reason that no model can 
fit the data is that the strong clustering on large scales 
implies that DRGs must occupy very massive halos. But 
DRGs outnumber these halos by a factor of ~20, which 
is only possible if individual halos host a large number 
of DRGs. This would mean that each DRG has a high 
probability of having several neighbors in the immediate 
vicinity, leading to a very prominent small-scale excess 
in «;(#). 6 As can be seen, the observed excess is much 
smaller than expected. 

6 Specifically, the amplitude of the one-halo term depends on 
the secon d factorial moment o f N occ (Mf l ), which we parameterize 
following Bullock et al. (2002). Note that this particular choice 
does not affect the main result of this section because, for the high 
(N occ ) value found here, it is generically expected th at N occ (Mf l ) 
follows a Poisson distribution for a fixed (e.g. Zheng et al.1 

2005); this fully specifies the moments. 
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6. DISCUSSION 

We have used the UKIDSS-UDS to perform the first 
precise measurement of the clustering of red, if-selected 
galaxies at 2 < z p hot < 3. These DRGs show strong 
angular clustering that is well-described by a power law, 
but with an excess at small scales. We use photometric 
redshifts to deproject the angular clustering, finding the 
spatial correlation length ro = 10.6 ± 1.6/i _1 Mpc. This 
value is comparable to that measured for lumino us red 
galaxies in the local universe (|Zehavi et al.l [2005h . how- 
ever DRGs are significantly more numerous. We show 
that standard models of halo occupation statistics are 
unable to simultaneously reproduce the observed cluster- 
ing and number density, because DRGs outnumber their 
inferred host dark matter halos by too large a margin. 

The most obvious explanation is that we have used the 
incorrect redshift distribution in deprojecting the angu- 
lar correlation function. A narrower distribution would 
reduce the correlation length (while a moderate shift in 
the overall distribution makes a relatively smaller differ- 
ence). However, a narrower redshift distribution would 
also decrease the effective volume probed by our sam- 
ple, thereby increasing n g . We illustrate these effects 
in Figure [3j which shows the observed ro and n g com- 
pared to the range of values for a typical HOD model. 
It also shows how estimating N(z) directly from the un- 
perturbed photometric redshifts — which, as mentioned 
in § El represents the extreme assumption of no random 
photometric redshift errors — affects the results. It may 
still be the case that we are subject to systematic red- 
shift errors, but we also note that a significantly nar- 
rower N(z) would adversely affect the reasonably good 
agreement between our e stimate of n a and the lum inos- 
ity functions derived by iMarchesini et~al . ( 2007a). Fi- 
nally, we have verified that our basic results hold when 
using a different photometric red shift code (HYPERZ 
iBolzonella. Miralles. fc Pell6ll2000D an d with a different 
template set ( Bruzual fc Charlotll2003[ ). 

Given the apparently high quality of our photometric 
redshifts, as well as the consistency with previous results 
for the clustering of DRGs, it is worth considering alter- 
native explanations. One possibility is that current HOD 
models are too simplistic, and that massive red galaxies 
occupy halos in unexpected ways. The fundamental as- 
sumption underlying these models is that galaxy observ- 
ables depend on halo mass, and not on the larger-scale 
environment or on halo properties such as structure or 
age. But environment may play a role, for instance via its 
effect on mass accretion rates (S cannapicco fc Thackerl 
120031 iFurlanetto & Kamionkowskil 12009 ). Additionally, 
halo clustering varies with several halo properties, even 
at fixed mass; this phen omenon is genericall y referred to 
as "assembly bias" (e.g. lGao fc Wnitdl2007t ). To the ex- 
tent that these properties affect galaxy observables, they 
will also affect galaxy clustering measurements. How- 
ever, we note that current estimates for the strength 
of the assembly bias are too small to a ccount for the 
observ ed discrepancies. As an example, iGao fc W hite 
(2007) have found that, at z ~ 2 — 3 and in the relevant 
halo mass range, halos in the upper 20% tail of the dis- 
tribution of halo spins have a ^20% larger bias than the 
mean value. It might then be supposed that DRGs oc- 
cupy less massive and more numerous halos with higher 




Fig. 3. — Correlation length versus number density. The black 
data point shows the values calculated using the preferred red- 
shift distribution (inset: black histogram), while the red data point 
shows how the results change when using a narrower redshift dis- 
tribution (inset: red hatched histogram). For the purposes of this 
figure we have fixed the power law index of the correlation function 
to 7 = 1.5, and assigned conservative 50% uncertainties to the ob- 
served galaxy number density. The dashed line shows the approx- 
imate relation for HOD models of ma ss-sel ected galaxies based o n 
the simulations of Zheng et al. (2005) and Kravtsov et al. (2004). 
The filled circles show the approximate relation for dark matter 
halos; the larger open circles are labeled with logfiWjj/M©). 



spin. But the increased number density of these low mass 
halos is approximately cancelled by the requirement that 
DRGs can only occupy 20% of them, so the discrepancy 
in number densities is unchanged. 

The existence of massive red galaxies at z > 2 
was not predicted by models of galaxy formation, al- 
though prog r ess has been made on this fr ont (e.g. 
Croton et all 12009; |Pe Lucia fc BlaizotJ l2007l but see 
Marchcsi ni fc van Dokkuml I2007bl) ~ The results shown 
here suggest that the conflict between models and ob- 
servations extends to the relationship between galaxies 
and dark matter halos. However, clustering measure- 
ments are sensitive to a number of systematic effects, 
so our conclusions remain tentative. The most obvious 
source of error comes from our use of photometric red- 
shifts. Ongoing medium-band NIR observations will sig- 
nificantly reduce the pho tometric redshift uncertainties 
(jvan Dokkum et aT1l2008f K and in the longer term multi- 
object NIR spectrographs will also improve the situa- 
tion. If future work confirms our results, clustering mea- 
surements such as those presented here may provide a 
new way to understand the detailed relationship between 
galaxy and halo properties. 
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